Section: Scientific Foundations

Lighting and Rendering

Participants : Mahdi Bagher, Cyril Crassin, Isabelle Delore, Olivier Hoel, Nicolas Holzschuch, Fabrice Neyret, Charles de Rousiers, Cyril Soler.

The classical approach to render images of three-dimensional environments is based on modeling the interaction of light with a geometric object model. Such models can be entirely empirical or based on true physical behavior when actual simulations are desired. Models are needed for the geometry of objects, the appearance characteristics of the scene (including light sources, reflectance models, detail and texture models...) and the types of representations used (for instance wavelet functions to represent the lighting distribution on a surface). Research on lighting and rendering within ARTIS is focused on the following two main problems: lighting simulation and inverse rendering.

Lighting simulation

Although great progress has been made in the past ten years in terms of lighting simulation algorithms, the application of a general global illumination simulation technique to a very complex scene remains difficult. The main challenge in this direction lies in the complexity of light transport, and the difficulty of identifying the relevant phenomena on which the effort should be focused.

The scientific goals of ARTIS include the development of efficient (and “usable”) multi-resolution simulation techniques for light transport, the control of the approximations incurred (and accepted) at all stages of the processing pipeline (from data acquisition through data representation, to calculation), as well as the validation of results against both real world cases and analytical models.

Image realism

There are two distinct aspects to realism in lighting simulation: First the physical fidelity of the computed results to the actual solution of the lighting configuration; Second the visual quality of the results. These two aspects serve two different application types: physical simulation and visually realistic rendering.

For the first case, ARTIS' goal is to study and develop lighting simulation techniques that allow incorporation of complex optical and appearance data while controlling the level of approximation. This requires, among other things, the ability to compress appearance data, as well as the representation of lighting distributions, while ensuring an acceptable balance between the access time to these functions (decompression) which has a direct impact on total computation times, and memory consumption.

Obtaining a visually realistic rendering is a drastically different problem which requires an understanding of human visual perception. One of our research directions in this area is the calculation of shadows for very complex objects. In the case of a tree, for example, computing a visually satisfactory shadow does not generally require an exact solution for the shadow of each leaf, and an appropriately constrained statistical distribution is sufficient in most cases.

Computation efficiency

Computation efficiency practically limits the maximum size of scenes to which lighting simulation can be applied. Developing hierarchical and instantiation techniques allows us to treat scenes of great complexity (several millions of primitives). In general the approach consists in choosing among the large amount of detail representing the scene, those sites, or configurations, that are most important for the application at hand. Computing resources can be concentrated in these areas, while a coarser approximation may be used elsewhere.

Our research effort in this area is two-fold: first we develop new algorithms for a smarter control of variance in Monte-Carlo algorithms, hence reducing the total cost at equivalent accuracy; secondly, we develop algorithms that specifically suit a GPU implementation, which brings us a huge gain in performance at the expense of controlled approximations.

Characterization of lighting phenomena

One of the fundamental goals of ARTIS is to improve our understanding of the mathematical properties of lighting distributions (i.e. the functions describing light “intensity” everywhere). Some of these properties are currently “known” as conjectures, for instance the unimodality (existence of a single maximum) of the light distribution created by a convex light source on a receiving surface. This conjecture is useful for computing error bounds and thus guiding hierarchical techniques. Other interesting properties can be studied by representing irradiance as convolution splines, or by considering the frequency content of lighting distributions. We also note that better knowledge and characterization of lighting distributions is beneficial for inverse rendering applications as explained below.

Inverse rendering

Considering the synthetic image creation model as a calculation operating on scene characteristics (viewing conditions, geometry, light sources and appearance data), we observe that it may be possible to invert the process and compute some of the scene characteristics from a set of images.

This can only be attempted when this image calculation process is well understood, both at the theoretical level and at a more practical level with efficient software tools. We hope that the collective experience of lighting simulation and analysis accumulated by members of the project will guide us to develop efficient and accurate inverse rendering techniques: instead of aiming for the most general tool, we recognize that particular application cases involve specific properties or constraints that should be used in the modeling and inversion process.

Example applications include the reconstruction of 3D geometry by analyzing the variations of lighting and/or shadows, or the characterization of a light source from photographs of a known object.